This invention relates to an image processing method and apparatus that enhance the sharpness of digital image while maintaining the blurred sense, that is, the blurring quality or lack of detail. The invention particularly relates to the technical field of a digital image processing method and apparatus that can not only suppress the graininess or other noise component of photographic or digital image but also enhance their sharpness while maintaining the necessary blurred sense.
In digital imaging technology, image data acquired either by recording silver halide photographic image or otherwise image with an image scanner or by making use of photoelectric devices is subjected to image processing and the resultant image is outputted from an image output printer. In order to restore the considerable deterioration in image sharpness caused by the scanner and the printer or to achieve intentional image sharpness enhancement, the edge portions of the recorded object (or subject) is usually enhanced by a so-called xe2x80x9csharpness enhancementxe2x80x9d technique, which is conventionally performed by use of a Laplacian filter or unsharp masking (USM).
In the conventional sharpness enhancement process, the entire portion of the image is subjected to the same processing and not only the principal object but also the object or objects as the background of the principal object which has become blurred on account of the taking lens system is enhanced in sharpness to produce enhanced edges. If the original image contains the background which was intentionally recorded in a blurred state, the final image comes out differently from what was intended by the photographer. The object image as the background rendered or the background image which are blurred by the taking lens system, that is, the image having the necessary blur (blurring quality or lack of detail), is not reproduced faithfully but the unnaturally sharpened image having no blur is reproduced.
Such an unnatural enhancement of the edge portions of the object as the background is not the only problem with the conventional sharpness enhancement process. The graininess in the image as well as the noise component in the image data due to the electrical noise from the scanner as a device for inputting photographic image are enhanced to deteriorate the quality of the final image.
This second problem was successfully addressed in Japanese Domestic Announcement (kohyo) Nos. Sho 57-500311and57-500354, as well as P. G. Powell and B. E. Bayer, xe2x80x9cA Method for the Digital Enhancement of Unsharp, Grainy Photographic Imagesxe2x80x9d in the Proceedings of the International Conference on Electronic Image Processing, Jul. 26-28, 1982, pp. 179-183. According to the method proposed by Powell and Bayer in these references, suppression of graininess is accomplished by smoothing (with a low-pass filter) and sharpness enhancement is performed with an unsharp masking (high-pass filter). In the smoothing process, signal values for nxc3x97n pixels are multiplied by Gaussian or other type of weights such that the signals are smoothed to suppress graininess. In the sharpness enhancement process, image signals for mxc3x97m pixels are first used to determine differential values by calculation from the central pixel towards the surrounding pixels and if any differential value is smaller than a preset threshold, the pixel of interest is regarded as representing graininess or noise and removed by coring and the remaining differential values which are greater than the threshold are summed up, multiplied by a constant greater than 1.0 and added to the previously smoothed signals, thereby achieving sharpness enhancement.
However, if a blurred object as the background is processed by this method, it is enhanced in sharpness, particularly along the edges, to produce an unnaturally sharpened image lacking the necessary blur.
The present invention has been accomplished under these circumstances and has as a first object providing an image processing method for enhancing the sharpness of photographic images recorded by a silver halide photographic system or a digital still camera, characterized in that the recorded principal object is enhanced in sharpness while, at the same time, the blur caused by the taking lens system in the object or objects that is not the principal object or objects as the background of the photographic image is recognized and kept as it is to produce a natural image retaining the photographically necessary blurred sense.
A second object of the invention is to provide an image processing method which, when performing sharpness enhancement, maintains the blur of the background and suppresses, rather than enhances, graininess and other noise component in the image, thereby producing a natural photographic image of high quality that retains the necessary blurred sense and involves a reduced amount of graininess and other noise component.
A third object of the invention is to provide an image processing apparatus for implementing the method provided by the first object.
A fourth object of the invention is to provide an image processing apparatus for implementing this method provided by the second object.
In order to attain the first object described above, a first aspect of the present invention provides an image processing method comprising the steps of: applying sharpness enhancement to original image data to generate sharpness enhanced image data for a wholly sharpened image; performing edge detection on the original image data to determine edge intensity data for distinguishing an edge region of an object and the other region in an image; determining a blur retaining but sharpness enhancing coefficient for enhancing sharpness of the edge region of the object but retaining blur of the image in the other region of the edge region of the object based on the edge intensity data; subtracting the original image data from the sharpness enhanced image data to generate a sharpness enhancing component; multiplying the sharpness enhancing component by the blur retaining but sharpness enhancing coefficient to give a blur retaining but sharpness enhancing component; and adding the blur retaining but sharpness enhancing component to the original image data to generate processed image data for an image enhanced in the sharpness while retaining the blur.
In the first aspect, the other region of the edge region of the object includes preferably a blurred image region in which the blur of the image is to be retained.
Preferably, the blur retaining but sharpness enhancing coefficient has a value equal or nearly equal to 1.0 in the edge region of the object, slowly decreases with the increasing distance from the edge region of the object and has a preset value of from 0.0 to less than 1.0 in the blurred image region.
Preferably, the blur retaining but sharpness enhancing coefficient is expressed by the following equation (1):
CBS(x,y)=b0+(1xe2x88x92b0)EV(x,y)xe2x80x83xe2x80x83(1)
where CBS(x,y) is the blur retaining but sharpness enhancing coefficient, EV(x,y) is the edge intensity data which is normalized (0xe2x89xa6EVxe2x89xa61), and b0 is a blur region sharpness enhancing coefficient for adjusting degree of sharpness enhancement to be applied to the blurred image region which is other than the edge region of the object.
In order to attain the second object described above, a second aspect of the present invention provides the image processing method which, in addition to the image processing method of the first aspect, further comprises the steps of: smoothing the original image data to generate smoothed image data; subtracting the smoothed image data from the sharpness enhanced image data to generate an edge/noise component including both a sharpness enhanced edge component of an object image and a sharpness enhanced noise component; determining a noise region weighting coefficient from the edge intensity data, identifying a noise component based on the noise region weighting coefficient and the noise/edge component and producing a noise suppressing component that introduces small variations in the noise component; and scaling the blur retaining but sharpness enhancing component to generate a first scaled component, adding the first scaled component to the original image data, scaling the noise suppressing component to generate a second scaled component and subtracting the second scaled component from the original image data, thereby creating the processed image data for the image that retains the blur but which is enhanced in sharpness and has the noise component suppressed.
In order to attain the third object described above, a third aspect of the present invention provides an image processing apparatus comprising: a sharpness enhancing unit for applying sharpness enhancement to original image data to generate sharpness enhanced image data for a wholly sharpened image; an edge detecting and computing unit for performing edge detection on the original image data and determining edge enhancement data for distinguishing between an edge region of an object and the other region in an image; a blur retaining coefficient computing unit by which a blur retaining but sharpness enhancing coefficient for retaining blur of the image in the other region of the edge region of the object is determined from the edge intensity data; a blur retaining but sharpness enhancing component computing unit for producing a blur retaining but sharpness enhancing component from the original image data, the sharpness enhanced image data generated in the sharpness enhancing unit, and the blur retaining but sharpness enhancing coefficient determined in the blur retaining coefficient computing unit; and a blur retaining but sharpness enhancing unit for adding the blur retaining but sharpness enhancing component produced by the blur retaining but sharpness enhancing component computing unit to the original image data, thereby creating processed image data for an image that retains the blur but which is enhanced in the sharpness.
In the third aspect, the other region of the edge region of the object includes preferably a blurred image region in which the blur of the image is to be retained.
Preferably the blur retaining but sharpness enhancing coefficient has a value equal or nearly equal to 1.0 in the edge region of the object, slowly decreases with the increasing distance from the edge region of the object and has a preset value of from 0.0 to less than 1.0 in the blurred image region.
Preferably, the blur retaining but sharpness enhancing coefficient is expressed by the following equation (1):
CBS(x,y)=b0+(1xe2x88x92b0)EV(x,y)xe2x80x83xe2x80x83(1)
where CBS(x,y) is the blur retaining but sharpness enhancing coefficient, EV(x,y) is the edge intensity data which is normalized (0xe2x89xa6EVxe2x89xa61), and b0 is a blur region sharpness enhancing coefficient for adjusting degree of sharpness enhancement to be applied to the blurred image region which is other than the edge region of the object.
In order to attain the fourth object described above, a fourth aspect of the present invention provides the image processing apparatus which, in addition to the image processing apparatus of the third aspect, further comprises: a smoothing unit for smoothing the original image data to generate smoothed image data; a noise region weighting coefficient computing unit for determining a noise region weighting coefficient from the edge intensity data; an edge/noise component extracting unit for subtracting the smoothed image data from the sharpness enhanced image data to generate an edge/noise component including both a sharpness enhanced edge component of an object image and a sharpness enhanced noise component; a noise suppressing component computing section for identifying a noise component based on the edge/noise component and the noise region weighting coefficient and producing a noise suppressing component that introduces small variations in the noise component; and a blur retaining, sharpness enhancing and noise suppressing section for scaling the blur retaining but sharpness enhancing component to generate a first scaled component, adding the first scaled component to the original image data, scaling the noise suppressing component to generate a second scaled component and subtracting the second scaled component from the original image data, thereby creating the processed image data for the image that retains the blur but which is enhanced in the sharpness and has the noise component suppressed.